1. Statement of the Technical Field
The invention concerns image processing, and more particularly, to an image processing method for calibrating images having different resolutions, for example, spatial and spectral resolutions.
2. Description of the Related Art
In the field of remote image sensing, two common types of images include panchromatic imagery and multi-spectral imagery. Panchromatic imagery is imagery that is obtained by a remote sensing device with a sensor designed to detect electromagnetic energy in only one very broad band. This one very broad band typically includes most of the wavelengths of visible light Panchromatic imagery has the advantage of offering very high spatial resolution. In contrast, multi-spectral imagery is typically created from several narrow spectral bands within the visible light region and the near infrared region. Consequently, a multi-spectral image is generally comprised of two or more image data sets, each created by sensors responsive to different portions of the optical spectrum (e.g., blue, green, red, infrared). Multi-spectral images are advantageous because they contain spectral information which is not available from a similar panchromatic image. However, multi-spectral images typically have a lower spatial resolution as compared to panchromatic images.
It is often desirable to enhance a multi-spectral image with the high resolution of a panchromatic image. In order to achieve this result, it is known in the art that the images can be combined or “fused”. In general, there are two key problems that must he solved in order to accomplish this fusing process. The first problem concerns a need for registration of the two images. The registration process involves a determination of where each pixel in the panchromatic image maps to in the multi-spectral image. This process must generally be accomplished with great accuracy for best results. For example, it is desirable to for each pixel in the pan image to be mapped to the multi-spectral image with an accuracy of less than 0.1 panchromatic pixel radius. A number of conventional methods exist for achieving this mapping. The second problem that must be overcome when performing the fusing process is to ensure that the radiance values of the fused image remain consistent with (1) the original multi-spectral image and (2) the original panchromatic image. Item (1) requires some means of obtaining an estimate of the weights that should be applied to radiance values for pixels associated with each band of wavelengths in the fused image. If these weights are known, then it is possible to make an accurate comparison of the radiance values of pixels in the fused image to the pixels in the original panchromatic image. Item (2) requires some means of determining the spatial distribution of weights that should be applied to each pixel contained in a region of the fused image which have are collectively mapped to an area corresponding to each pixel in the multi-spectral image. If the spatial distribution of weights is known, then it is possible to make an accurate comparison of the radiance value in the fused image with the radiance values in each band of the multi-spectral image. Such an evaluation can ensure consistency of radiance values as between the original multi-spectral image and the fused image.
Conventional algorithms utilized for performing the image fusion process suffer from several limitations. For example, they generally make simplistic assumptions about the manner in which the high spatial resolution pixels in the panchromatic images should be combined or fused with the high spectral resolution pixels of the multi-spectral image. Typically, these include (1) an assumption that high spatial resolution pixels from the panchromatic image down-sample to the low spatial resolution of the multi-spectral image as a box average; and (2) an assumption that the pixels from the panchromatic image are evenly weighted averages of red, green and blue spectral bands.
Some algorithms have also adopted a slightly more sophisticated approach with regard to the process of down-sampling pixels from the high resolution of the panchromatic image to the relatively low resolution of the multi-spectral image. For example, in some algorithms, the high spatial resolution pixels from the panchromatic image are not merely down-sampled as a box average. Instead, a point-spread function (PSF) is used to determine the manner in which the high spatial resolution pixels from the panchromatic image are down-sampled to the pixels consistent with the multi-spectral image. The PSF (sometimes referred to as instrument line shape) is a characterization of the manner in which a point of light is blurred or diffused by a sensor and its associated optics. Accordingly, knowledge regarding the PSF of a sensor can be useful for down-sampling the high spatial resolution pixels from the panchromatic image to the multi-spectral image. In particular, the PSF can be used to define a weighting system for combining individual ones of a plurality of high resolution pixels to form a single larger pixel at lower resolution. However, the PSF in conventional algorithms has merely been approximated based on sensor geometry data, such as aperture, focal plane resolution, and so on. As such, the PSF that is used is not necessarily an accurate representation of the true PSF for a sensor system. Further, a pre-computed PSF will not contain information specific to a given image pair such as residual shift error in the registration and artifacts of image preprocessing.
In view of the foregoing, there is a need for an improved method for obtaining an accurate estimate of the weights that should be applied to each band of multi-spectral image data to approximate the panchromatic spectral response. There is also a need for obtaining an accurate estimate of the spatial distribution of weights that should be applied to a region of pixels in the panchromatic image geometry that map to a given multi-spectral image pixel. This spatial distribution of weights can be advantageous to approximate the multi-spectral spatial response when fusing lower resolution multi-spectral image data with high spatial resolution panchromatic image data.